Papers with human-computer interactions
ChaI-TeA: A Benchmark for Evaluating Autocompletion of Interactions with LLM-based Chatbots (2025.naacl-short)
Copied to clipboard
Shani Goren, Oren Kalinsky, Tomer Stav, Yuri Rapoport, Yaron Fairstein, Ram Yazdi, Nachshon Cohen, Alexander Libov, Guy Kushilevitz
| Challenge: | Large Language Models (LLMs) have revolutionized many NLP applications. |
| Approach: | They propose an autocomplete evaluation framework for LLM-based chatbot interactions that includes a formal definition of the task and suitable metrics. |
| Outcome: | The proposed framework evaluates 11 models on a task that performs fairly but still lacks the ranking of the generated suggestions. |
Cards Against AI: Predicting Humor in a Fill-in-the-blank Party Game (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Humor is an inherently social phenomenon, with utterances shaped by what is socially and culturally accepted. |
| Approach: | They propose a dataset of cards Against humanity, including 785K unique jokes, and train machine learning models to predict the winning joke per game. |
| Outcome: | The proposed model performs twice as well as random on the more difficult task of judging novel cards, with the context having little impact. |
Corpus Design for Studying Linguistic Nudges in Human-Computer Spoken Interactions (2022.lrec-1)
Copied to clipboard
Natalia Kalashnikova, Serge Pajak, Fabrice Le Guel, Ioana Vasilescu, Gemma Serrano, Laurence Devillers
| Challenge: | linguistic nudges can influence people to the same degree as a human agent, according to Thaler and Sunstein (2008). |
| Approach: | They propose to use a corpus design method to compare influence between linguistic nudges with positive or negative influences and three conversational agents: robot, smart speaker, and human. |
| Outcome: | The results show that linguistic nudges can influence participants to the same degree as human agents. |
Characterizing Similarities and Divergences in Conversational Tones in Humans and LLMs by Sampling with People (2024.acl-long)
Copied to clipboard
| Challenge: | Existing taxonomies or text corpora suffer from experimenter bias and are not representative of real-world distributions. |
| Approach: | They propose an iterative method for simultaneously eliciting conversational tones and sentences . they run 50 iterations with human participants and GPT-4 and obtain a dataset of sentences and frequent conversational tone. |
| Outcome: | The proposed method can be used to characterize the differences between humans and LLMs. |
Can LLMs Learn Uncertainty on Their Own? Expressing Uncertainty Effectively in A Self-Training Manner (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit excessive, random, and uninformative uncertainty rendering them unsuitable for decision-making in human-computer interactions. |
| Approach: | They propose an uncertainty-aware instruction tuning method that aligns LLMs’ perception with the probabilistic uncertainty of the generation. |
| Outcome: | The proposed method improves LLMs' performance by 45.2%, with reasonably good out-of-domain generalization capabilities. |